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One of the most popular approaches to satisfying this feedback ';equj^ement is to 
S^deotape the s/udent in interpersonal situations Most such eff^ts. owever. 
^all quite short of providing appropriate feedback ^^J the .general/fa lure^^^^^ 
demonstrate training benefits mm the use of this method can be (Attributed to 
SrshJncomlSg. Further, the costs and difficulties associated\wii;h prOVfdin^^ 
fdequatrfe^b^^ interpersonal performance apparently ri^ave discour/y^ 



aged attempts to do so. 



This report presents an approach for decreasing "some of.these^o^s and diffi- 
cu tieir This approach' uses a computer to assist in the.process>f elicitinc, 
appropriate feedback.from judges and providiri^^this ^teck tojhe stud^^^^^^ 
both iVcoordihation with the videotaped recordings of the student s perfomanee 
In thiWegard. computer progranvs designed to help a J^^ge ^ate -interpersonal _ 
perfomance along a variety o« dimensions vvere developed and tested^^^ 
thesfe programs also are suitable for providing dback to the stud^i^. for^^^ 
tests of this aspect of the process were not conducted in the-present research. 
* nrll! albeH bHef . experimental evaluations of the- feedback .el icitation and 
*(^opment |^rog rams were conducted, however. 

JeYeral. potential benefits of the prbceiittres for inte':PfSonal skills training 
WrI Semoastrated as a result of both the. formal and informal tests of the 
^yst,em'. ' 
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. . . ' . ' V • ^ ' . -PREFACE - 

Thjis project wqs jointly funded by the O^ffice of Naval Research and the 
J'. Navy PeVsonnel Research "and Develo'pnent Center.. . " . 

• ' ' Matlfy people at Recrui't Training Cotmiand (RTC),. Orlando vere very helpful 
in this project; most notably CTRCS Paul; Zetter holm. and J.CD8 William Sullivan, 
a^.well as all of the Company Commandeirs who participated -in the videotapes; 

' • . Pa,t Smith was' responsible fqr^^^puter progratmiing, system operation, ^the 
eol'lection of experimental da ta /|^d marty helpful suggestions concerning. the \ 
design of the program material.^;^ 
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'.- . ' , . ' ■ ' . SUMMARY > .• .' t ' ■ 

Jraining technology in the interpersonaH skil 1 s area has not kept pace 
with the training'technology that is avail ablfe in most^pther sl<ill areas. The 
deficiencies in interpersonal skills training are especially apparent in "job . 
oriented", instruction, wherein the student perfoH^s operational tasks in . simu- 
lated." or -a ctuaO job situations.^ A major obstacle to the application of job 
orfented irtstruction to this area is the difficul-ty in providing appropriate 
feedback to {fte student concerning his performance in the interpersonal situa- 
tion. ■ . . » ^ ' ^ ' • 

^ • . ■* 

One of the most popular approaqhes to satisfying khis -feedback requirement 
is to videotape the student in inte*:personal situations," Most sucfy efforts , 
however, fall quite short of providing ^ippropriate feedback and the general 
failure^to demonstrate training benefits from the use of .this method can be 
attributed to this shortcoming. Further, th€^ costs an^K/dif f iculties ass9ciate(j 
with providing adeqtiate feedback regarding interpersonal performance apparenily, 
have discouraged attempts to do^sp. I 

This report presents an a'pproach for decreasing some of these cost? and. 
difficulties. - This apprS^ch uses a computer to as si s.t in the prcfeess of 
eliciting appropriate feedback- from^ judgeS^s^ providing this, fee^dback to the 
student, both in coordination with the videertaped recordings of the student's ♦ 
performance.: In this regard, computer programs^desfgned to he^lp a judge rate- 




the present research. Formal, albeit brief,,- experiment^ evaluations of the 
feedback e>icitation and developrjient ^/o^jrams were cprfduV^ , hQvpver. , 

^ ^ ■ / ' - J . ■ ^ • ' ' 

Several potential benefit's of ,the procedures yor^inter personal skills * 
training were demonstrated as a result of hotht)^ foifjpial^and Informal tests', 
of the system. First, the tasks of el iciting^^eveloping , Bnd delivering feed- 
rback a^Dp^red 'to be facilitated by the programs. Evidence for this came mainly 
'.from the e)speriences of project per sonneVV/ho-^ noted, in developmental tests of ^ 
the system; tha.t\these training t^fk^sTwere difficult, frustrating, and some- 
times even impjjssible to perform correctly wfthout computer assistance^ Also, 
inrthe formal evaluations of the ctHriputer-assisted methods, judges reported 
th&t the work load was. "atiout right;" that the task was interesting, and tha^^t 
' they understood the, task. ' ; \ ■ 

' ^econdjthe feedblack that was made possible by this new tefchi^que^ appeared 
to be much more useful tl\ajn the feedback that typically results Jmm other ap- 
proaches in the area/ This conclusion was- based on anStnalytical comparisdT^ 
of" the extent t6 which the feedback of the new programs vs.* that of other, non- 
computer-aided methocis, conformed to the characteristics of good feedbe^ck* ^ 

Third, the computer progranjs can be ysed to improve the reliability of * 
ratings. of interpersonal performanqe andi>in so .doing, improve theNvalfdity of 
the ratings. Altl^ou^h, for.a variety .ef possi^l e reas^s* overall rating 
reliabilities were relatively^ low, a newN^putef-aid'ed technique improved ttie 

■ . ■ ■ . - • ■ .' I- 



Reliability on one of the two varia'&les (considerate) to which It. was appl ied.. 
This technfique provided computer-genferated, t^me-weighted averages of a series 
of individual ratings-of a variable. The reliability resulting from this' 
metho'd was-^consider^ly greater than a global rating made by the judge on the 
variable. c- ; • , 

The pre^liminary nature of-^his. study, prevented extensive refinement of 
the^prograins. and the procedure for their use. : For example, the judges were 
not exteh^iyely trained in the use of the compTrtet;^ rating metho>ls, and little^ 
was done t!) as«wj;^[that thfey^per formed conscientiously. Also,- more should be > 
done to a§,5ure that>th^ functions .and fo^itiats of the programs are appropriate. 
These 1 att^r. improvements await more extensive development amf research 'ef- 
forts. These 'efforts shogld^resuTt" in.a more compl ete ^fK'o^ram of computer 
. aids, aft^kwhich a more extensive effort to appl^^i'ajicT^vaTuate the programs^ 
•shoulid b^iconducted.. ' * ^^^^\i>^^'4 ^ 4 ^ i^f' 

The present stud^ demoostrate'cl ^hesprobarb^ty't>iat/a variety of computer 
proc^*am» da'n. enhance intenpersorjal skill's trairung. The nature of ideal com- 
puter progrjams for this purpose i-s a, subject fgr further research. 

Further research is also needed to determine whether students perform any 
• differeritly (hopefully better) a^,a re'su-lt'of this'new feedback ^procedure, ^ and 
how the$e (hanges compare to those occurring under more traditional (i.e.,'non7 
xomputerj) 'raining methods. This training effectivehess issue (i.e^^, how well 
does th^ t ^aini\ig chan^^e performance) should be addressed after additional 
developften : and refinemeiit of the new technique. THe concern about training 
validity (■ .e.^, how well) does^ the training improve perforrpancte) is 'a much more 
thorny [pniblem, since universal agreement on t-he characteristics and measure- 
. ment of gond leadership does not exist, despitf? a^g'reat^ body of research on 

the subjec- . The new technology presented here is'^intended tp^be content free. 
^ That is, ii should teach the subject matter of /interpersonal instruction more 
efficiently, regardless of the nature or quality o(*f this si/bject matter. More 
. efficient training strategics are necessary, but not sufficient^, conditions;, to 
improve interpersonal [farformance^^ Improved. perfo*mance in this area also 
requires va^l id subject matter,, and this concern was only ancillary to the^ 
goals of th^ present effort. / ^' ^ - 

■ \ • • ^ ^ ^- • r 
Fin2^11y, once the training effectiveness and validity issues have been . 
resolved] cost-effectiveness issues arise. . Cost-effectiveness^q^estions re- 
quire compare' sons of the new technique. with other logical approaches, to the 

^ objectives. It was beyond the scope of .thi s study to develop^and evaluate 
al ternarelaFproaches. For this reason, and because the training effectiveness 
and validity issues have yet to be r&splved for the new tectinique, rigorous 
Efforts t6 answer cost-effectiveness question|/^are premature. Nevertheless, 

~ assuming the desirability of the functions of the new Jcomputer techniqcres, and 
comparing jthsse techniques with^tourely maniJal operatio,'ns for performing^ 

P^ryfe functions, considerable cpst^savings can be .demon'strated. ^ ^ 



V 



FRir 



t 




^ - •. NAVTRAEnuiPCEN IH-308 



\ 



, ... TABLE OF CONTEI^TS 

Section ■ ; . ' . ' , ' ' r , . 

. - \ . 'preface'.; .-. . . . . . . , . . . r . . . . .. V2 

/ • SUMMARY ...... \ '. . .■ . . y ^' ' ' 

I INTRODUCTION. . . » . • > - 7 

- 11 ' GE^^tRAL METHODOLOGY . . ...... \. .• 11 

Rating Variables. .• . u . . . . .. • • • . 

f Experimental Apparatus. ... . . . . .*. . . . . , * ^3 

f - ^ . " ; ■ . 

MI PRELIMINARY STUD! . . . . . .• . .\ . , . . . 14 

/ \ ■ ^ , O - - 

Rating Conditions . ..'....*.. ^ .... _ U 

Videotapes. ...... 4 .... ,15 

* Procedure ... 15 

Results . ..... . o • '•'•^ ' 

. IV FOLLOW-ON STUDY . .v • ' .' • . . 17 

New Rating Cor^djtion. . .« . . . *. . .• . . \. . . . ,17 

Procedure . T . . .V 

Results 37 

' ' ' ' r ' • ^ iJ? 

V DISCUSSION •■ • .• • • • •' ■• • • • • • • - • • • • 

REFERENCES . . ,\ . 23'. 

APPENDIX A - Instructions to 'Raters ; '25 ' 

APPENDIX B - Role^l ay Scenarios'. ,. .-^ . ...... ... 30 

ARPEND.IX C - Questionnaire. .. .\ ...... ^ . [V 



APPENDIX D - Sample Computer Oiitput^. • • • • • • 



» ■ ■ ■ ' 

^- ' ■ - ; . :• ^ ■ .... 



ERIC .^ ^ 



NAVTRAEQUIPCEM rH-308 

\ .. ) 

SECTION I , ' 
^ . INTRODUCTION 
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The efficiency of an orgainizatiort^^ d the effectivGness of inter- p 

personal comnunications within^it. TypiciilTy, tittTe is done, to improve per- 
fonpance in this area. i / < * , ,> , t 

" . .-it" ' ■ • ' • . . • ' - 

The Naval Training Equipniejit. C^^^ during , the past five 

^years, has been engaged Irt det'e?^ and evaluating several ap- 

''proaches to the improvement Vf irrterpersonal behaviors trirough the use of a 
cTDmputer. 'Tfjesa approaches iave mainly, been an'^^^ at improving the inter- 
personal communTcations abil ities of Navy recruit company coimanders^ (C'Cs^ in, 
interactions with their recruit^/ Jhese ^fearlier approaches are described in 

other reports J-W.5.6.7;8.9;. . \ M ' • 

One of these approaches involves tPt* use of videotapes to teach inter-- ^ 
personal skiTU. The -computer controls the operation of a videotape ptayer, 
coordinating *trte tape with written material presented on a CRT terminal . The 
tap^ consist of both real, and rple-played interactions between CCs and re-,- 
cruits. The lessons train and test the ability to recognize examples of vari- 
ous interpersonal skilly. These videotape lessons follow an extensive set of 
written CAPlessons which cover the same skills. The video lepsons wfere 
designed trf; present the skills in a form'which more closely approxim^artes the 
-real operational setting.' » ? 

1^ Spencer, G. J. and Hausser, D. 'L/, Blaiwes, A. S. and^Wel ler,. D. ' Use of 
Computer-Assisted Instruction for Interpersonal Sfill Training - A Pilot Study, 

1975. Technical Report: NAVTRAEQUIPCEN 73-C-Ol 33-1 . , ' ; 

2. Cohen, J. L. and Fi shbein f;!^ Development and Research Utilizing :the 
PLATO IV System for Company Commander Behavioral Change Tr*aining. August .1975. 
Technical Report: NAVTRAEQUTPCEN 73-C-Ol 29-1 . > ; • > 

3. Rausser, D, L. , Bl^wefs, A. S., Wel ler, D.^ ^Rv, anpd . Spencer, fi. J. Appl ica- 
tion of Computef-Assis}fceci; Instruction to Interpersonal Skill training. tJanuary 

1976. Technical l^eport: \NAVTRAEQUIPCEN 74-C-OlOO-l . ' ' 

4. ^ Cohen,. J. L./ and FishbeinAM. A Field Test of the PLATO. IV Sy<5tem fof 
Company Commander Behavioral Chpnge Tra;^*ning. Cluly!1976. Technical Repoftr^ 
NAVTRffi^QUIPCil^ 744C-0095-1 .- • ' . v 

5. Lukas, G. , Blallwes, A. S., ^nd WelTer, D. R; Evaluation af Human Relations 
Training Programs.^ January 1977. Technical Report:* NAVtRAEQUIPCEN 
75-0-0076-1. • . ■ ' ' , ' ; . . , . 
&i Blaiwes, A'. S., Weller, ^D. R., and Rom6£, G^ Development and Implementa- 
tion of a Computerized Evaluation and' Training System (Cr^) at A Recruit 
Training Command. Mar^h 19^78. Technical- Report: NAVTRAEQUIPCEN IH-300. 

7. i^eller, D. R. andeBlaiwes, A. S. Leadership Dimensions of Navy Recruit 

Company Commanders ancRRecruit Morale and Performance. * Psychological Reports, 

1976, 39, 767-770^ ^ ^ 

^. Blaiwes, A. S. a.nd Wei ner, D. R. A Social Simulator: Devel(^ent and ♦ 

Eyaliiation.^ Jduca^nal Technology. March^l577, 14-20. ^ 

9. Blaiwes, A. ^and Welleff D. f^. A Computerized Evaluation and Training 

System (CEtS) for Recrui,t Training Commands: An Overview;' NAVTRAEQUIPCtN, .in' 

prepar^tMbn*) ** . - ' 
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The current research is .direeted at Isnl arging the use of v1deOtat)e in the* 
' training of interpersonal skills. An ide^l videotape training program wouVd^ 

Svtart with the taping of a student in an actual job situation. The tape's, 
- febntents^would then be rated for the performance of the inter persona V-sM T1 

Th6lf the student wtfJd.be given feedback on his/her performancei^^v^th^eci^^^^ 
^ attention to strong and ^yeak areas. 'Additt^nal training would tfterl he given . 
' oB^appropriiE^te skills. cThe taping - feedback - training cycle be re- " 

p6ated until performance was .acceptabl e. ' - 

■ ■ ' J' r' : ■ •• V/'' • 

\ A, critical step in the'de\telopment of thi s program is the design of -a ^ 
.rnethod for accurately, efficiently, and rel iab^ly Vat ing. interpersonal, behaviof • 
^ as seerypn /videotape. In thq effort to develbp a better method for rating 
j - ifitertje^ aspects of the feedback component also were pro^ 

7 videcl. ^These are the sabjects of this s-tudy. . ' - 

v; It was decided at the outsort that a /computer would be used to assist in 
the' rating. Some-of th^ more apparent advantages of using a computer are: • 

■ •. ^ . ' ■ ■ . I -'^--r- ■■■■ "■:•'/■•■ 

a. . Rater^*^ responses can be automatical ly keyed to the; location on the 

; y?- tape where they occur. ' ' ^ r ^ ^ - 

< • » , ■ • ^ ' '> 

b. Raters' responses can be' automatical ly $tpred--and processed. ^ ^ . 

c^ , Complex outputs agd displays can be quickly produced 

d. All functiQjhsf of the tape player can b^ under programmed control . 

\i ' e. The m^t effective -approaches to videotaped, rating canr'be standardized. 

^Tfie need:^r research and dWelopmgftt (R&D) on the pr^ocess of rating, 
analyzing^ and' providing feedback ori a student 's^videptaped performance is 
indicated b^y t he widespread practice W using videotapes for performance feed- ; 
bacl^y^^in training programs, contrasted with the relatively minor experimental 
eviderice-whi ch would Recommend t^^ei^:• use. ^ 'This is esR^cially true in the 
interpersonal skfl Is area. ' ^ " ' . ' ~ \ ' / 

Most evidence in favor of such use o"f videotapes appears to^be non-quanti- 

I 10 » * 

tative and anecdotal. For example, Kanner reports that instructors become 

vividly aware of their shortcomiligs as a result of viewing their performance 
on videotape which "..Vis often followed by^^striking changes in the instrut- 
tor's teaching behavior. .'..i These insights are Aften achieved much mor§ 
.rapidly than when someone eJse, yrithdut the tjenetn of videotape playback, . 
criticizes ain instructor*s"perfbrmance. " ^A similar observation 'was made in an 
effort to compare videotape and rqle-playing procedures for teadiing job ihter- 

J viefe skills. It was noted "...that seyeral subjects in vthe role-plarying 

,. ^ ^ • ■ 

Kanner^J. K. )n a Reporf^f the Third -Armed- Forces Television Con-fer- 
^nce^ rt)rtLee, VA, V7-T 9^ October 1962. Office of The Chief Signal Officer 
Audio-Visual Commmiications Directorate, April 1963, p. 14. 
* 11.. Venardos, W. G.^^^and Harris, M. B. Job Interview Trai^iing with Rehabilita- 
tion Clieriis: K Comparison of Videotape and Role-Playing Procedures. Journal 
Q Of Appli^ PsycWlogyT ISySj'^vol. 58, No. 3, 365-367.^ 
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groups initially "resisted the prfssibility of their -having interv.ieiw'problems,. 
wheV'eas those: in. t>ie 'videotape group wer'e quick to d^tiect their CJWn inappro-' 
priate behaviors and acknowl edge the need *f or improvement. " ^ 

; More (Quantified 0*^^^^ to recommend the use • , 

of Vidpptapes so strongly. For example, videotaped performance feedback did 
no|: lipprove group performance in^a problerrt solv^g situation when^used alone; 
although 'it .did add smal 1 iqiprovements when, corgbined with a ^videotaped model 

' \o • 

of^ performance. Simil ar results . were obtained* fn. an experiment which tested 

the effects of model and feedback videotapes on teachers' use of positive feed- 

' • ' • ' ■ 1 3 ' ' ' ■ ' 

back'.y^jesponses tQ chil dren 's^behavior. "The findings demonstrated that the 

Qombijied model* and feedttack^idieotape technique, and the model videotape tech - 

.niquevare effective while the feedback videotape intervention is ,^ineffect'ive 

in changing teaching behaviors." \ ■ ,^ • ' 

y ■ . . ^» . ^ . . ■ ' * ■ ^ ' . ' .' 

other* researchers, noting ffie cost of videotape procedures, t?econmend 
other methods'^to achieve the training goals, For.example, when , no difference 

jwasifound between role-playing and videotape procedures for the job int^eryiew . 

'training (both being superior to an Attention-placebo control), the aiithors 
concluded, ^with some reservation, that: "While videotape is' being used in- . 
creasingly.::today -for t^ching varioiis interpersonal skill s^ such as inter- / . 
viewing/, tiie saviiigs in equipment' expefise that would result froiti. empl oying. the * 
equally ef.feative procedure of role-playing might sug'gest that role-playing be; 

.. ■ ■ 14 ^ ' '•■ ■ -v- . 

the treatment of choice/" ", - 

\ ' Another effort to suppled operational procedures for training instruc- 
tors a.t: Sfi&ppard Air Force-.Base with videotaped* performance feedback resultqci 
in conql-tisions as follws: "That those capabilities have potential payoffs is; 
assumedV. N't it is nqt^clear how those capabilities can properly "be applied.... 
Use of >tfe^^ as a supervisory, tool in the existing ischool at, Sheppard Air 

•Foree. Base, offer no r>et advantages. Tha payoffs do not warrant the cost."^^. 

5Qftia k'ey woVds in-,this 1 atter"*"quotation are ".T^haw . . . ^(the performance'^ 
feedback Videotape ). can be properly applied." Little seems to be dbne to im- 
prove.vtrtj^ ■^^ videotaped experien<?e beyond merely allowing the stu- • 

dent ^to view the tape, arid^even l;ess seems to^^^known abtout what can be djlne 
to enhanc* the^f fectivqness^^of such training. . ' , I ' - 

^ ,. i ' ^ ■ ■ ■ , / ' . ^ . , ^ . - ■ 

12. Walter, ^G. .Effects . of Training Inputs on Group Performance. Journal 
of Applied' Psychology, 1975, vol. 60; No. 3, 308 -'31/2^ j. 

13. -Rutherford, R. B. The Effects: of a Model Videotapf and Feedback -Video- 
tapes on the Teaching Stales of Teachers i»n!T raining. The Journal of Experi-; 
inental. Educatrion.- Fall t973^;:..v9l 42, No. .T, 64-69. , , w . . , 

14. - See fgotnate* 10 .on ;page 8.' , ^ • - 
,16V: Hays, j.^'.and Pull"iam, R. D^velopiTient and Evaluation of Video Systems for 

'Performance Testing and Student Moni tori-ng.^ Technical Training Division, 
Lowe ry Air Force" Base, CO, AFHRL-TR-74-67 , July 1974., p. 169; . 
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•■ - ' ■* SECTION II . ' " ' - • 

. '. GENERAL HETHODOLORY , • • - 

■Three important factors in the-design of an interpersonal skills rating 
method are: (1) reliability, (2) ease' ^nd convenience of rating, (3) useful- 
ness of resulting ratings. The mean r ^ "t two items' are fairly ob- 
vious,, but the third requires son^p ^ - W principle purpose of -an 

••Interpersonal sicills rating of r -em with feedback on their m 

perform^/iee which -will Teid to\ c .nei performance. Theref 

i-the results of the rating procedu. t . ly understood by the CCs, . 

specif ic enough so that the "CCs will know what steps to take to improve thdit 

• performance. Also, the feedback must be valid.fsuch that a CCs performance 
will in- fact improve as a result of tak'ing the ^teps suggested by tfhe feedback. 
A second purpose of the ratings is tp.'provi.de .information tn management for . 
use in decision making and organizaffonal ^develc^pment. Thus, the outputs of 
the rating procedure miisf be suitable for this purpose also. ^ 

It was decided tfrSt a tW&step process woul(^ be used to design and^^test 
the interpersonal -Jl^vl 1 s • rat-ing^mettiod:.-^itbe-.r<y^.c5:t:._s.t€fi,-_zthcee=.^^ 
rating methods were' deviled. ,TfvSse methods were 'then testerl, using a small 
group' of raters. The three fa<ttoi:s mentioned aboVe isrere all used in evaluating 
these methods, in the second step, a hew rating rtiethad wa's devised, incorpor- 
ating the best features of the preliminary method^. A small group of raters 
also provided information for evaluating this metnpd. . y 



RATING VARIABLES V \ 

' \ ■ • ^ ' ' ' . ' 

The variables to be rated. were selected from t^hose isolated during earlier r 
phases of., the research. Al so. considered were performance variables empld^yed in . 
a variety of other 'interpersonal training and evaluation systems (e.g., / 

Klauss,2° Fleishman,^^ and Bowers^^),. The particular variables employed here 
were selected because of their commonality to many d,f these systems ^nd because 
-they appeared in pilot tests to be more amenable thah others to the present 
rating context. There we're three levels of variable^ chosen: glo-Bal , skill, 
and behavior. ^ 

GLOBAL LEVEL. At this level, the raters were to judge the overall effective- 
ness of the CC (how well the CC was meeting the goals \of the interaction). 

SKILL LEVEL. At this level, the raters were to judge \fie CCs nerfcjr.mance in 
two skill areas: \ ' ■ 

\ ■ * 

20. ■ klauss, Rudi. Measuring the' Impact on Subeirdinates of Managers' Inter- 
personal Comnunication Style.s and Credibility. The Maxwell School of Citizen- 
ship'^nd Public Affairs, Syracuse University, TR-2 , 1^77.^ 

21. Fleishman, E. A. Twenty Years gf Consideration and Structure in E. A. 

, Fleishnan and J. G. Hunt (Eds. ) Current Developments in the 'Study of Leader- 
ship. Carbondale*: Southern Illinois Univer-,ity Press, 1^)73. 

22. Bowers, D. G. anci Seashore, S. E. Predicting Organizational E^ectiyeness 
with a Four-Factor Theory of Leadership. Administrative Science- Quarter I y , -■ 
Vol. II, No. 2, Sept. 1 966, pp P38-263. ' ; 
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a. Considerate - To what extent does Jthe CC treat the%(iru1t like a 
human being? ^ . * * ^ ' . ^ ; 

• • . . ' . • ^\ 

^ Clear -^How clear is the CC's corrniunication. with the recruit? 

BEHAVIOR LEVEL. At this level, the rasters .were to judge-the CC's .perfonnance 
, ^of certair>'^behaviGrs. (both 'positive Snd negafiVe); The <six teen behaviors' 
'selected fall under positive and negative representations of the two skill - 
areas: V " . , - \ • 

a. Considerate - Positive' - 

(1) - -^The CC asked the recruit how he/she felt aDout the situation. 

(2) The CC expressed confidence in the recruit's abil-ittes. 

(3) The CC was patient with the recruit. 

name. 

^ b. Considerate - Negative 

(1) The CC showed disinterest' in the^ recruit' s' feel ings. 
^ (2) The CC belittled the recruit's abilities.'- , ^ 

(3) The CC was impatient with the recruit. 

(4) The CC used abusive names.' 

c. Clear - Positive ' V 

(1) The CC gave specific details. 

(2) The CC encouraged recruits to ask questions. 

(3) The CC asked questions of the recruits to vsee if they understood. 

(4) The CC gave reasons to show the relevance. of what he/she was 

saying. 

^ d. Clear - Negative . - '^ ; 

(1) The CC was vague. 

(2) The CC discouraged questions. 

(3) The CC failed to question the recruits foe the i^ understanding. 

(4) The CC failed to give reasons although the situation called* for 
it. . ' 
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EXPERIMENTAL APPARATUS \ y # 

This^exlDeriment used, a 'Nova 3/12 minicomputer, a Sony c^setfe vf,c|jgptaT3e 
pla3(er, a Sony monitor^ a Tektronix ''disVl ay terminal, and a Data General 
printer* The videotape recorder was connected tp the computer through a 
specially designedVinterface which allowed the computer to control all of the 
^ functions of the pwyer. The subjects receiv.ed instructions and feedback on 
the terminal > and made responses on' the terminal keyboard. * The printer .was 
used for data output. ^ ; ' • . .* 

Control of- the videotape ^ ]J|p^" was accomplished througti timing. * For 
example, it was necessary ^ 1^ ' ^ precise, spot 'on the ta^e which was 

' being shown when^ a subjer .pon^ Jfc that -the tape cpuld be* stopped 

automatically, at the samf e next slfflfWlng. ' To *do this, the computer 

kept^ track of the elapsed th . the seginent when each response was. made. 

On subsequent shoxings, when this elapsed time was reached, the computer would 

' automatically activate the/'pause" mode on the player, causing lihe tape to 
stop. When the subject completed his* inputs at this spot, a terminal keypress 
would cause the- computer to activate the "play" mode, and the tape would 
continue. ' ^ , ... ^ . ^ 
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♦ ■ ^ . . SECTION IH V ' 

• ' .„ PRELIMINARY ^"TUDY ' 

« The purpose of the prel iminary study was to deterntine v^♦Tich of the candi- 
date rating methods'developed. for/ this study, or ,which features of these 
methods, werelto be retained tor 'further study.- . " 

•RATING CONDITIONS^ > . - 

* * V. ■ ■/ ■ 

• A number of^ifferent methods fbr rating interpersonal skills with com- ^ 
puter assistance were conrp-'^rl. These jaig^re narrowed to three candidate ^ 
methods which utilir - which seemer ^ Dromising and 

vhjch could -be in . e resources ^ ^ Some prom- ^ 

ising featlires weit .loi included at this time ...^ - .iiMidtiOMs ir ':"^e experi- 
mental equipment. For example, random access to tajJe segments would require 
signals from the tape player to. the corriputer, artd at present the signa^ flow • 
is only'from the computer to the tape pUyer. ythe reverse is, ^ however, tech- 
.nically feasible. Each of the three methods yielded scores on the two skills, 
bot different' procedures were used to arrive at these scores. Eacfi method also 
yieltled scores on various other rating variables* 

MEfHOD A. While viewing a \qdeotape of an /nter'^ction between a recruit ^nd a 
CC, the rater made a continuous rating of the overall- effectiveness of the CC. 
This was done by pressing keys 1-5 on the Computer terminal (1 ? totally in.-. . 
effective, 5 = very effective) whenever the rater noticed a change in the over- 
fall effectiveness of the CC^s performance/. The computer calculated a time- 
weighted mean for these ratings in the overall effectiveness -category, which 
comprised the "computer overall" score. Following this, the rater viewed the 
same tape segment again. This time/, the tape stopped af each point v^iere -the 
rater K:hanged ?iis/her rating duringVth^ first viewing, and the rater was a^ked 
to report whether 'he/she had observed- the CG being clear or considerate during 
the prior segment. The rater also had an opportunity to type in a commenV^t 
this time. Following this viewing, the computer displayed the total number of 
times each skill had been pbserved and the rater'mad^ a summary evaluation for 
each skill', as well as a summary, evaluation of overall effectiveness (providing 
the "judge clea>", "jqdg'e considerate", and "judge overall" scores). 

'method B. While viewing a video^^ped interaction between ? CC and a recruit, 
the rater made a continuous rating of the-CC's performance on one of the two 

"skills (by pressing keys 1-5). The computer provided time-weighted means for 
these scores, resLjlting in the "computer clear" and "computer considerate" 
scores. Following this, the rater viewed the same segment again. This time, 
the tape stopped at each point where the rater changed his/her >ating during 
the first viewing, and the rater was -asked to select the behaviors which he/she 
had seen during'the previous segment (from the eight behaviors under the skill 
in question). At the end of this viewing,, the total number of observiations of 
each behavior was displayed, and the rat&r made a summary-rating for the skill. 
Following this*, the entire procedure was repeated for the other skill. Final- 
ly, the rater made an overall effectiveness rating. 
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METHOD C. While viewing a videotaped interaction betwefjn a CC and -a recruit, 
the rater pressed^ a key on the terminal each time he/she T)bseryed one of the^ * 
16 behaviors^ Followingr this the rater viewed the. same segment a^ain. This, 
time, the tape' stopped at' each point where the rater had marked a behavior; 
The rater then , selected the behavior which- he/she had- s^en. Following this 
viewing, ,the computer displayed ^'he total number of observations of each be- 
havior, and the .rater entered a summary sqore for each ski'lT. FinallK* th£ 
rater-entered an overall effectiveness rating. ' ' . 

The Bctual. instructions to the raters are in Appendix A'. • ' 

VIDEOTAPES V ; • ; . - ^ . . :^ 

Thfee'wkTeotape segmer^*'". we»*fe selected for the .picel.in1in<^»^y study. Each 
involveJKa^C intera.ctin( '^.Tncj,l,€'''r:ecruTt "^r^ *-,hp segments was a" 

real-l ife interaction, i -^^^ ; vht- .wo were roje-p ^rCu ^r..,> Jons, wfth two 
CCs playing the parts of C and recruit. 7 The foTe-pT?^^^ 

Appendix B. The first two scenarios are the' orgies actually used in the study. . 
-Sevenal- mi:e:=rpl^^^ 

Al 5J0>in eluded ifc Appendix B is an example of a^ behavioral checklist, v^liich 
acconjpfinies the second role-play.^ Such checklists were used' in CC schoal 
classes to evaluate role-players and stimulate class discussion! / The check- 
lists were provided only for maximizing the^ benefits of the /ole-plays-in CC 
school classes, and were not otherwise involved in the present .study. Various 
personnel at, the Recruit Training Command, Orlando viewed all, of the taped 
segments and verified that the. content and performances were realistic. 

PROCEDURE ' . ' • 

- ■ . »^ 

The subjects 'were six students (two female, four male) from Florida Tech- 
nological University. • All of these students •were psychology majors with some' 
experience with behavioral rating techniques. Each subject used each of the 
three rating methods. ^*A given tape wasValways rated by the same method by al-l' 
subjects because representing all method X tape segment conditions would have 
resulted in only two subjects per condition,^ which would be insufficient to 
compare the response characteristics (where the tape is stopped, what ratings 
are given at these points, etc.) of different subjects under similar conditions 

■ • 
Following each method, the subjects were given a questionnaire on their 
opinions of that ffTethod. The questionnaire is reproduced in Appendix C. Each 
subject was also interviewed ifi depth by the experimenterso 

• Outputs from the computer provided a record of all .subject responses. In 
addition, the computer provided mean ratings for methods A and B, and graphical 
displays of subject respoases for all methods. Sample outputs are contained ' 
in Appendix D. ^ . ^' ' 

'RESULTS ■ " ' ■ ' ' 



The results of>.he preliminary study are organized according to the three 
factors mentioned earlier; reliability, ease of use, ancj usefulness of ratings. 
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sRELlABILITY.^ The standard deviation of each score was used as aii. indication of 
reliability: Thfe smaller the standard deviation, the greater the .agreement 
across judges » and, therefore, the greater the reliability. Th^ standard 
• deviations ire preseritedv^n Table 1. Method G showed* the highest variability 
on the three .measures", which were common to alT methods >( judge considerate/ 
jM<toe clear, judge^overall).v The variabilities G(f method? A a1nd B were c^ii^e 
similar. M \. , ' ' ' - 

EASE OF USE. Data on ease of use, came from the opinion questionnaire, items . 
T-3. On all three -i tens (work Ipad, understanding, interest) t)^ere were no 
significant or iinportant differences across methods. The ovewl means wete: 

- WOrk"in;^H - (";»hniit right"); iiriri_P]r_<;J;a nH ing - 1.4 (betwfeen "not -at ,al 1 COn- 

fused" and "slightly confused"); jnter^st - 2.8/("sl ightly interesting") . - On • 
the .item "which, method dolyou prefer," two subjects chpsa A', two chose B', and 

{ two chose C. Both subjects 'who chose A stated that the cy)tion to type in 
cbrnments vyas^ '^sjor fa'ctt)r in th?^^^^ choice . _•_ * ^ 
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USEFULNESS OF RATING\^ All three methods provided ratings by the judge pf the 
wo ski T l^dH^o-ve^a H-^f f ee^^^ s B a n d C uT pro v^dnd^T^sts^t-^ 

the betiaviors observed.^ ATI of this should be useful feedback to CCs. Method 
B provided a coraputep-processed, time-weighted average score for each of the 
'two skills, whil§>^eth6d A provided such a score for overall effectiveness. 
These cpmputer -processed scor^^would serve as useful checks on the judges' 
opinions. ' . V ' , 
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TABLE 1. SCORE 


STANDARD DEVIATIONS 






VARIABLE • . ^ ' 


METHOD 
A 


METHOD 




METHOD 
C 


Judge considerate 


.55 


.51 . 




.81 


Judge clear^ • / 


- . .21 . 


^ .-33 


1 


.87 


Judge dveral 1 




.> . 


.93 


Computer considerate 




■ A ■ 




» - 

* 


Computer clear 




\ .51 , 







Computer overall ' , 
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* " ' • ' ' ' ^ 

SECTION IV - ' -"A • / 
FOLLfiW-QN STUDY ... : ' ' . ' 

NB< RATING CONDITION ' ' ■ • / . . " .- "* 

\ ' " - 

Based on the results from the prel iminary study, a new rating* method was 
designed. It was mo^ similar to method B, in that skills-were rated first, 
followed by sel ection of beliwviors. The .method of selection of behaviors f>r§t 
(method C) was rejected because of its low reliability. Methojd A was not 
favored because it did not yiel/the 'behavioral istings, which were considered 
to be usefi|iil. However, the comront option of method A was retained. 

In the n hjo, ji M^jt^ i jri^i viewed the segment without rra^^iny 

^responses. The tape, was tjien reshown, and the judge press.ed, a key oh/the 
Jterminal every time he/she saw- an instance of rf^conSiderat^^^^^^ This ' 

keypress st opped the tape. The^ judge' gave^ a^^^r^^ VeveT^ 
of considerate behavior at this point on the tape. Following this, the judge 

given ithe opportunity to type in any additional comments./ The tape then re- 
started, and the same process continued. 

-At the end of the segment, the judge entered an overall rating for con- ^ 
' siderate (1-5).' Then the tape was reshown, and the same procedure was followe-d 
for the skill clear. Finally, the judge entered an overall, global rating. ' ' 
Sample outputs for this new method are included in Appendix D. 

Pl^OCEDURE ' ^ . 

The subjects were three students (one female, two male) from Florida Tech- 
r?ological University. All of these Rodents w^re psychology, majors , and 
familiar with behavioral rating techniqueSo A. fourth subject participated, 
but experienced considerable difficulty with the experiment,' so the data' was" 
not used (the subject was foreign-born and had some trouble, with English.). The 
tapes used were the same as in the preliminary study. Each subject rated all 
three tape segments using the. new method. After rating the tapes, each subject 
filled out the first* part of the opinion questionnaire ("questions for each 
methoji")- 

RESULTS ' ' \ ' / • 

RELIABILITY. To assess the rel iabil ity of this method,^ the intraclass correla- 

23 * ' 

tion coefficient (as described by fiuilford ) was used. This is a measure .of 
the degree to which ^different raters agree on the rating of a given variable, 
across a number of ratees. . 'Table 2 presents the intraclass correlation co- 
efficients for all variables, alohg with the ratings. . t 

The reliabilities of the two considerate scores were low, with the com- 
puter-averaged ?core showing a higffer rel iab1l ity than the judges' sebre (in- 
traclass correlation coeffi^ents in excess of .90 are not uncormion in this 

JT, Guilford, J. P. Psychometric Metiiods. McGraw-Hill , 1954, 395-397. 
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■ ME 2. RATMitlWLASS. CORRELATION COEFFICIENTS 

SEGMENT 1 ' • • ■ . SEGMENT* 2 SEGMENT 3 INTRACLA'SS . 

RATER RATER' RATER - RATER RATtR RATER 'RATER RATER MR CORRELATION ' , / 

• iNsL I ■ 3" ; 1 ' 2 . 3 • ■ -1 l I COEFFICIENT S .', . 

Judge considerate. ''^.SO 5.00 4,00." '2.'00. "2.20 4 00 ' 1.70 4.30. 5.00 ' • .26' 

■ Computer considerate"2.8^ M8 3;72 '■ 2.12 3.93 3.95 , 1.B4 .4.24 , 3. ' 
,Judqe clear M.SO ^.00 M.3f) 3.00 ' flf :,i)U ..9U-_ 7 . 79 

.oi..;„:c'r cledi o./b ,t>.D0/3:96 ; 2.94 3.20 2.81 Z. 73 ' 5.00 3.37 -^8 '■. 

■ Judgeoverall ' • • 3^J0> /5.00.\4.0r' '-UO 1.50 .3.00 , 2.50 4.80 3.00 ,'.55 • I 
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There are tljr^j^f actors which might have contributed to the unimpressive 
.reliabilities. First, the raters "were given 1 ittle tratning and practice in 
the use of the method. It is reasonable to assume that more training in the 
use o'f^the method and the meaning of the variables wouTd improve the reliabilp 
ity of the ratings. This is especially true of considerate, wjiich has a' less 
obvio-us meaning than clear., ' f, 

A second factor contributing to low reliability isVthe lack of variabilit\ 
.in. the performance of the CCs who were taped. All of the CCs were attempting ^ 
to do their best, and they naturally avoided any obviously unclear or incon- 
siderate behaviors. No attempt was made to role play a "bad" CC.. The^ ii^jii- 
sion of sf cl^MHy bad performance w^ increased the variance . of -both ' 
variabi es, and' aTmost certafnTy woUT ~ 

- FinalTyv chararteristics" of; the ratinfj^-variabTe (su^ch ^5 the 
ease of applying It) would affect its reliability. The%nphasis in the current 
study was on how to facilitate the prpcQ-ss of judging and providing feedback on 
interpersonal performance, re^ardles^ of the particular, aspect of performance 
that is being judged or fed back. Thus, relatively minor consideration v^s 
given to the basic reliabil ity of the rating variables in 4>his context. The 
.present techniques can, at best, allow the maximum rel iabil ities inherent to a 
rating variable to be. manifested in thp ratings. If these inherent , rel i abi 1 - 
Ities are low, the manifest reliabilities wi,ll also be low, no matter how the 
ratings are accomplished. For example, the computer time-weighted method con- 
tributed considerably to, the re-liability of the libnsiderate variable (the co- 
efficient increased from .26 to^.45). Thus, the computer method is considered 
to be of benefit to ratings of considerateness in spite of the relatively low 
reliability of those ratings. Studies are needed to determine rating variable 
<:haracteristics that are conducive to good reliability. . • ' 

EASE OF USE-. These results tome; from the opinion questionnaire. On the '^ork 
load" item, all subjects reported that the work load was/"about right." On the 
"interest" two subjects selected "somewhat' interesting" and one selected 

"very intere^ing\" On the "understanding" item, all subjects reported that' 
they were "not at all confused." Therefore, it wa's cohcluded that this method 
was quite acceptable as regards ease of use. 

USEFULNESS OF RATINGS. , This method included all important features of the 
three preliminary methods. It yielded a judge's average for each variable', a - 
'computer average for^each variable, a-n overall judge's average, a list of be-, 
haviors observed, and judge's comments. AIT three subjects freely used the 
comment option. All ratijigs and comments were keyed to the poi;nts in the tape' 
where the relevant behavior occurred as well as an oveVal 1 summary, at the end ' 
of the tape recording. The avai.lability of hard-copy printouts of all feedback 
information woulJenabT'S the student to compare hi^ performance acros^ per- 
formance sessions.* ^ • ^ 

It is therefore' concluded that this method would be very useful, both for 
feedback to CCs and managerial purposeSo ^ " > v 
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'section V 

. DISCUSSION . . • 

Previous quantitative research on the use of videotape as a feedback and 
evaluation tool, in the i-nterpersonal skills area has not generally yielded en- 
couraging resijlts> A possible cause of this lack of positive results is the 

.failure to develop sophisticated and/or ^tandardiz'edv methods for applying 
videotape in thf% area. ' The cu/rent research attempted to t^ike a step in that 

^direction by developing a te^Rnique for rat^'ng interpersonal behavior as seen 
"On videotape, v/ith the aicf of a computer. / - ^^ 

The technique designed qn this study provides detailed information on 
interperfonal performance which' should be highly useful in a v^iriety of wys. 
The most obvious use is that bf/provid^ing perfomance feedback to the subjects 
of the tape. This- technique appears ito be simple to use, and should not re- / 
quire extensive rater training. One questionable area is that of reli'abTlity, 
and it is likely that further development woyld improve the reliability o^his 
technique substantially. Further', the improved rel iabil ity of the coTisi derate 
variable as a result pf the computer technology provides encouraging evidence 
that aclditibnal R&D on these issues can conifribute importanjtly tp the area. 

The contribution of the compliter to tTiis 'technique was a major /area of 
concern. If desirable judging and feedback objectives (as represented in this 
studyi could be achieved' equally well (or .better) without computer assistance*, 
the expense pf the computer would riot be justified. However, after considering 
ttie*pr6bat)le reduction in effectiveness and increment in Costs that would be 
associated with non-computer administration of the technique, it was decided 
that computer assistance is cost-effective. 

Th^ cost "of the computer equipment used in this study was approximately 
$30,000. However, this equipment performed many functions in.addttibn to those 
related to this study. It i.s.^ estimated- that a minimum system capable of ad- 
ministering the technique de^pribed here would cost less than $2,000. 

Using a non-computer technique, the rater would proceed in much the same 
manner as with the computer, except that he/she woul^d stpp and start the .tape 
playerr/nfenually, and record responses, on paper instead of typing them on a key- 
board/1 The major additional task would be to record the exact point on^the 
tape where the responses we^-e made. This would requiir*e the rater to consult a 
timer or counter at each po^nt where a response was made, and record the 
.reading. At the end of the rating process,- the £ime-weighted averages would 
have to be manually calculated. . * • ' , , 

This procedure is more .difficult to accomplish than it may first appear. 
The major source of difficulty is that the judge is burdened with a host of 
-"housek^ping" tasks (i.e., stopping the tape at the appropriate points,, 
recording timeSj calculating scores) at the same general time ^as vhet) the taped 
performance is ^o be rated. The process of rating interpersonal performance 
requires intense concentration 5uch that even minor additional duties can be 
seriously disruptive. ^ . ^ 
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Due to the early stage of development of -the" computer progr^ams and the 
lack of definition of other means for performing the tasks, no attempt wafs 
made to rigorously compare the various computer capabilities WT^th^other-, non- 
computer ^ethods.^^ Anecdotal evidence on this pointy however, was obtained 
during dwelopnentfal testing of the progrJims. In these tests, the present 
authors performed various rating and feedback tasks manually in order to eval- 
ud(te the appro priyateness of candidate programs for compute»2,.^,implefTientation, 
If became-<iuite apparent from these" efforts that. some tasks coul d not be 'Per- 
formed at all manually with only one operator^: and some of these tasks could 
not be perfotited satisfactorily even. with two operators, these tasks typically 
required sb.much time and concentration to perform that the.operator cpntinual- 
Jy lost his main train of thought and had to repeat previous operations to 
regain it. . . ^ 

The operations which wer^ most disruptive to the total task were the com- 
putation of rating scores and tbe-Cdntrol of videotape presentation. Also, 

these operations^^'were never performed as accurately manualTy-as-they--wer^^ - 

computer control- and their accomplishment was associated with frustration and 
tedium.^ These problems were largely resqjjved through use of computer support. 
Thus, the cpmputer programs increased the" 1 fkel i hood that the operators not' 
only would be abl| to propercl^y perform the tasks, buttaTso that they would bfe . 
willing and motivated to perform them properly. » ^ 

The greatest advantages of computer assistance in the current application, 
however, become apparent when one considers the process of giving feedback to 
a person on his/her rated performance. Although the feedback mode was not 
implemented (fusing this study, the provision of feed-back is the- pr'^incipal 
justification -for developing , the rating procedure.. With computer assistance, 
the ratee would watch his/her performance on tape; and the computer would auto- 
matically stop at e^ch point wherg.^the rater had made a response. The ratee 
would th6n be presented with a rating on the skill in question, a list of be-: . 
haviors noted by the judge as relating to that skilT, and any other ^^"^ / 
comments. ^, ' ^ 

Without c6'mputer control , the ratee 's attention to the tape and the feec^- 
back would be seriously disrupted by the need to attend to the manual operation 
df the tape player and timer, and codrdination of the delivery of the feedback. 
If another person were to operate the tape player, twice the man-hours would be 
required for the feedback process, and the added expense would soon outweigh 
the cost of computer control in any operational situation.' There is also a, 
privacy factor to consider, since the ratee might be more^ilT at ease with 
another person [iresent. 

Thus, the system -performed most all of the important feedback functions 
which would be performed by a skillful human instructor, .and at a lower cost. 
Irt addition, the computer system would perform these functions 'more accurately, 
reliably, objectively, frequently, patiently, comprehensively, and with grfeater 
potential for modification and impnoyement than ciould be reasonably expected 
from a human instructor. Thus, because human^- instructors cannot, will not and/ 
or do not properly perform these feedback functions, the present system would 
provide feedback 'that is far superior to that typically provided^ in inter- 
personal skills training programs*' 
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The Current computer capabilities, atifd additional capabiHtles suggested 
by this study", could provide the means for "impl^rientifig a yariety of other 
techniques for aiding the judging, process that would 30 well. beyond the com-' 
pTexity of this preliminary demonstration, THfe computer capabilities demon- ■ 
-strated or suggested in. this study provide flexibility tb explore potentially 
useful judging and feedback techniques which would be impossible to accomplish 
by other means. . \ ' 

Additipnal functions, not irriplemented in th^ current study, but v*iich the 
computer could readily perform, involve the automatic storage and -processing 0^ 
data on the performance of ratees. For ex-^mple, a ratee could be given feed- 
back on bow his performance compares with all other persons rated with the 
systen. Alternately, the ratings of several raters could be combined to pro- 
vide feedback from different sources. Or, the pfercent of time that a ratee 
exhibited- a given characteristic could be computed. Another fancti.on which 
was briefly mentioned, is random access to tape, positions. The tape. player 
could be made 'to send position information to the computer, which would allow 
the computer to locate any point on the tape automatically (using rewind and 
fast forward modes, as. well as play mode). As an example of the use of this 
function, a ratee could choose to review only those portions of a tape v^ere 
he/she was rated as being inconsiderate. The computer could control the player 
such that only these segme'nts were showir^ fast forwarding between them so that 
intervening material could be 'dipped. 

In conclusion, it se em apparent that further research oh the videotape 
rating technique developed in this study is warranted, and that the use of a " 




/ 




4 



BlaiwBS, A. S. and We 

■•^ System (GETS) for 
/ , irv preparation. 

Blaiwes, A. S. and We 
tion. Educatiprra 

.Blaiwes, A. S. Welle 
* of a Computerized 
Training Command. 

Bowers, D. G. and Sea 
with a Four-Facto 
Vol. il. No. .2, S 

^jEohen, J. L. and Fish 
Company Comrtianfler 
Report: NAVlfWEQI 

Gotten, and Fish 

'' System foih Compan 
tecfjirtical Reporf: 

Fleishman, A.y Twe 
Flei shnian and 0. 
Leadership. Garb 

Guilford, J; P. Psyc 

•* ' * .- ■ 

. Hausser , P. L. , Blaiw 
of Computer-Ass i^ 
1976.' Technical 

Hays, J.'. and PAjlliam, 
Performance Testi 
Lowe ry Air Force 

^Kagan, N. .(Can. Techno 
'Interaction?^ Edu 

''kanner; Jv H. In a R 
Fort Lee, VA, 17- 
Audio-Visual Comm 

Klauss*, RCi,di. Measur 
Communication Sty 
and Public Affair 

Lukas, G.', Blaiwes, ^At 
Training Programs 
75-C-0076-1: 




ERIC 



, - . . , • NAVTRAEQUIPCEN IH-308 

Murfllin, J. A Report on the Fifth' Armed Forces Television Conference, 

Sheratbn-PaH< Hotel, Washington, DC, 2-3 November 1964. Office of the 
Chief of Communications-Electronics 'Pictorial and Audio-Visual Directorate, 
June 1965, p. 49.^ ^ 

Rutherford, R., B. The Effects of a Model Videotape and Feedback Videotapes 
on the Teaching Sytles of Teachers in Training. The Journal of Experi- 
. mental Education. Fall 1973, yol . 42, No. 1, 64-69. . > - ' 

Spencer, G. J. and Haus^er/D. !• ; Blaiwes, A. S. and Weller, D. R. Use of 
Computer-Assisted^^Instructiian for Interpersonal Skill Training ^ A Pilot 
, Study. 1975. Technical Rep^hj;: NAVTRAEQUIPCEN 73-C-0133-1 . ♦ 

^Venardos, M,.G. and Harris, M. B. Job Interview Training with Rehabilitation 
' Clients: A Comparison of Videotape and Role-Pl aiding Procedures. ''Jdurn'al 
of Applied Psychology, 1973, vol. 58, No. ^, 365-367. . 

Walter, G. A. Effects of Training Inputs on Group Performance. /Journal of 
, Applied Psychology, 1975, val. 60, No. 3, 308-312. " . v 

Weller, D. R, and B.laiwes, A. S. Leadership Dimensions of Navy Recruit 
Company Commanders and Recruit Mourale and Performance. Psychological 
Reports, '1976, 39, '767-770. 



. ■ \ ' . NAVTRAEQUIPCEN IH-308. , > ' 

• APPENDIX A - ' . ■ 

; V . . ' > INSTRUCTIONS TO RATERS ' 

-INTROOUCTiON ' 

In tWs study, you will be' rating the perfonnance of persons in inter- 
personal situations (as seen'on videotape)., The persons you will be rating 
are Navy Company Conmanders. ■ It is their job to lead companies of 80 new re- 
cruits through eight weeks of basic training. The sections of tape which you 
will see involve interactions betwfeeri. Company Commanders (CCs) and their re- 
cruits. Some of these interactions are role-played and some are taken from 
real-life situations. 
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•VARIABLES 

the variables on which .you will be rating the CCs " 
fall into three. levels: ' " 

1. 'OVERALL EFFECTIVENESS-LEVEL - How effective is the CC in interacting with 
the recruit? • , . ' 

2. SKILL LEVEL - How well does the CC perform certain skillsTy^The skills you 
will be rating are:; / ' • ' j * 

CONSIDERATE - -To what tx tent does the CC treat the recruit like a human ' 
/ being? ^ \ 

CLEAR - How clear is the CCs coirriuni cation with the recruit? 

3. BEHAVIOR LEVa - flow well is the CC performing certain behaviors? The 
behaviors which you mll-^Jae looking for fall under the two sk-i 1 Is above: 

. CONSIDERATE - POSIT 

1. The CC asked the recruit how he/she felt about the situation. 

2. The C£ expressed, confidence'in the recruit's abilities. 

3. -The CC was patient with the recruit. 

4. The CC used the recruit' sname. 



CONSIDERATE - NEGATIVE 

1. The CC showed disinterest in the recruit's feelings. 
2.. The CC belittled the recruits abilities. 

3. The CC was impatient with the recruit. 

4, The CC used abusive names. 5 ' 



CLEAR - POSITIVE , 



• , 1. - The CC-gave specific details. . 

2. The CC encouraged recruits to ask questions. 

3. The CC asked questions of recruits to see if they understood. r. . 

' 4. The CCigave; reasons to . show the relevance of what he/she was saying. 

' clear - 'NEGATIVE - - . 

1. TJie CC was vague. - , . . > 

2. The CC discouraged questions. 

y 3. The CC failed to question the recruits for their understanding. 

4.. The GC failed to give reasons' although the situation called for it. f . 

Please stud^ these variables so that you.'wi IV be 'familiar with them as you 
rate.the -tCpesJ .Ke^ this sheet handy to refffV- to. . , 
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• RATING METHOD A ,' . " ' 

I ■ ' . 4 ■ 

You will see one of the segments which you viewed previously, in which ona 

CC, inter^acts with..two different recruits. As you watch, you will make a 

continuaT rising of the OVERALL EFFECTIVENESS of the CC's performance. 

You will db this by pressing keys 1 - S on' the: terminal i 

. ' Key T - 'the CC is totally ineffective , 
Key 2 - the^CC is somewhat ineffective 
Key'3 - tKe CC is neither effective nor ineffective ' 
Key, 4 - the CC is somewhat effective ^ f f 

' Key- 5 i:the CC is very effective \f 



or example, if you feel that the CC is **somewhat effective" at a given 
point in the tape, you will press 4. The cdmputer will . continue to record 
a rating of 4 until you press another key. You should change your key 
Selection every time your^opinion changes. The tape 11 start with key 3 
in effect. Make'' your first response when you have>enaugh' information to 
decide how defectively the CC is perfoy*ming. Do not press any keys while 
the recruit^k^speaking. 

Ypujwill watch the segment again. This time t\k tape will stop at each 
(point Where you changed your raiting in the previous viewing. When it ^ 
stops, you will asked if your change in rating v^s caused by the CC's 
performance in the CLEAR or CONSIDERATE area. You uifill also have an. . 

opportuni-ty to type in a comnent on the CC's performance. ^' , v 

. ■ ' ■ \ ■ , ■ « ■ . » 

After the second showing, the computer will display the number of times 
you saw each skill. You will then give the CC a summary rating on each 
sk'ill, as well as an overall rating. 
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- . • . RATING METHOD;fi . ' ' 

1, You will see one of the segments which you viewed previously. As you 
> watch", you will majke' aj£pn,tinual .rating of the CC's performance on one of 

" the tv« skills. You wm. do this "by pressing keys 1 -'5 on the te'rminal, 

" ' ' ■ ■■■ ■ * 
Key 1 - the CC is performing very poorly on the ski'll 
Key '2 - the CC is .performing poorly on the skill „ V/ 

, - Key 3 - the CC is performing ?at an average level on the skill / 
Key 4 - t/he CC is performing wel 1 on the skill ' " . / 

Key 5 -^.•the CC is performing very well on the skill y' 

• For example, if you see that the CC i-S performing well on the skill at a/' 
certain. point on the tape, you will press 4, The computer will contipW 
to record a rating of 4 until you press another key. You should change)/ 
your key selection every time your opinion changes. The tape, will start 
with key 3 in effect, ' Make your first response when yoii have enough ;in- 
formation to decide how effectively the CC is performing. Do not press 
any keys while the recruit is speaking, / 

2, ' You will watch the segment again. This time, the tape will stop at -each 

point where you changed your rating in the previous viewing. When it 
stops, you will select the behaviors which caused you to change your rating^ 

3, At the end of the second viewing, the total number of'dljServatiojis of each . 
behavior wfll be displayed. You will then give the CC an overa)l rating. > 
on the skill, - • 

4, Steps 1 -3^11 be repeated for the second skill, • ''^''^^ 

5, You will tive the CC an overall effectiveness rating, ^. V \ 
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. RATING METHOD C t ;^ - ' ' 

\ou will se^ one of the segments whjch you v'iewed previously. As you 
watch, you will press the space .bar every time you notice one of the be- 
haviors bfeing performed. Keep the'^list of behaviors handy so that you can 
refer to it. The tape will not stqp during this run-through, ^ 

You will watch the segment again. This time the tape will stop at each 
point, where you pressed the space bar. When it stops, you will select 3 
the behaviors which 70U observed. 

' ' . ' • . •■ - - ' ■ 

At the:, end of the second* viewing, the total number of observations of each 
behavior will be displayed. You wilt then gjKre the CC an overall rating 
for each slgll. ' ^ ^ 

You will give the CC.^an overall effectiveness rating. 
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* .• . ^ APPENDIX B . 

' • ROLE-PLAY SCENARIOS 



ROLE-PLAYING EXERCISE 
. COMPANY COMMANDER 



It is the 4-2 day of training. Recruit, Wilson has been having serious^ 
problems throughout training, mainly with MED inspections. It has been decided 
that Wilson will ga to T.U. 4013 (fold and stow). Wilson will>go to the 
Division office at»0800 l^omorrow; then will return to the barracks, pack his. 
(her) gear, and go to T.U. 4013. Generally, recruits stay«»^in T.l). 4013 for 
two to ten days, then go to a T.U. which is at the day of training where they 
left tiieir original l.'V. . 

y:-f' ■ • - \ ■ - . i • ; ■ ■• • ■ 

^ ^Recruit Wilson' has never been a disciplinary problem. Wilson has a'Tways 
tried hard, but seems a little "slow" and has been overwhelmed by all of the 
things that need to be le'aj^ned at RTC. It is '\ikely that,^with the extra . 
training in T.U. ^4013, Wilson will be able to make it through RTC. 

. * ' . ■ ^ ■ ■ . ■' 

Now ypu.need to tell Wilson what has been decided.' Think about what you 
want to say; and when you are ready, begin. 

. • „ ROLE-PLAYirfc EXERCISE ' . : 

, V /-^^ RECRUIT . "'^ - . * ■ > " 

You have just been called to the CC's office. You expect that this/ has 
-something to doVith your poor performance on inspections. You are very ' 
worried that you may be kicked out of the Navy, because you would really Tike 
xtoJttay in and make a career of i|:. ; . C 

^ • Figure B-rl . Role-Play #1 . • 
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' , , " ' • ROLE-PLAYING EXERCISE . 

' ^ RCPO 

\ '• ■■" • \; ' ■ ■/ 

You are an ROPO at RTC Orlando, . CPO Allen (substitute^^name of person ' 
playing CC rolfe)^*QS been^the CC in your company since its formation fouj 
weeks ago', • ^ ' , ! 

. ' Everything" seemed to be. gotng along pretty well for the first couple weeks j 
of training* • For'the past t^Q weeks, however, tW company has been fs(lling 
apart and the CC, won't -dp anything to *stop it, • 

The companjy'scbresi^on the last inspection (i^^e^ 8 discrepancies in ^: 
personnel, 13 in locker) were way dowi from ^e previous inspections (4 in 
personnel, 7 in locker); there has been an increase in street marks in the 
Jast week; and the.pPQ^had to establish order in the barracks twice in the 
last two weeks, V ' / ^ . 

Y(|U feel that these probVems are due mainly to tte attitude -and perform- 
ance oMhe CC, Recently, the CC is expecting you tjajgcto a Ibt of things that 
seem to bp his gob, 'You don't tfTink it's fair of . him^to expect you to run the J 
company practically on your pwn. Many problems are aris^ing that you don't have ^ 
. the time or ai)ility to handle. You're not sure what is expected of you and the 

- other R PCs feel the same way, ^ r 

: - . ' ■ ^ J , 

Many of the Recruits also are complaining about the mar\agement of the 
company. Most recruits agree that: (a ) the CC-stWuld be around more to watch 
over things; (b) he is not interested in the company anymore; (c) he is con- 
cerned only about winning' flags i^ith the least possible effort ff^ojn him; (d). 
"he- has little confidence in their ^Wlity to perform well; and (e) much of 
recruit tratninig is a Waste of time, . ^ ^ 

When you have tried to tell the CC about these pnoblems, he become(..?ngry, 
throws the blame on you, and shows no fnterest ifi* changing his ways, Ip-'is 
quite upsetting to be yelled at fnr problems that are the CC's fault,- especial- 
ly when you, have been ki lling yoursel f to run the company and the CC will .take ♦ 
no rfesponsibi+Tty for the probl ems, / ■ 

You have decided that. maybe >it's better not to talk to the CC about these 
. problans. You tune .him ^ out when^he blames you for the problems. Until he ^ 
shows that he wiH meet the probl ems more reasonably, you decided to pla^ dowi 
, your discussions with him. Jrtstead of talking with^him about the problems, 
you are sending as many of the problem cases as you -can to the CC. .This v«y; 
he will be able to.sde first hand^ what you're up against and it will 'force him *\ 
to become.more involved with the company a gai^h, ' 

However, you would like tb be able to discuss these inters with the'. CC. 
Thus, if (and only if^ he shows signs that you can trust Kim to be more reason- 
able and considerate wil 1 you tell him what you think;r 

f Figure B-2. Role- Play #2 •• . ' 
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- • ROLE-PLAYINR EXERCISE S - • 

A ( CC, , . - 

^ • You are a CQ at'RTC," Orlando. Bob Sanford (substitute name of person 
playing RCPO) has beep the RCPO in -your company since its 'formation four weeks 

>ag^.' :■ ■ »^"v * . ' ' *' ■ • ■ ■ ■ ■ . \, 

RCPO Sanford seemed to"be doing a satisfactory job for the first three^ 

weeks^of training. However, there h^is been spme evidence t^e past week Jo _ 
.suggest that the RCPO is not doing an\dequate ffob. This evidence was obtained 

from the performance of the Company and observations of and conversations with 

the RCPO and other members of the. company. . - \ 

The company scores on the lastMnspection (i.e., 8 discrepancies in 
personnel, 13 in locker) were way down frdm the previous inspections (4 in 
•personnel, 7 in locker); there has been an increase in street marks in the 
last jieek;-and the DPO had to establish order in the barracks twice in the 
last two weeks. . ^ V • 

. ' You feel that -these problems are due mainly to the. a.ttHudftS and perfp 
ance of the. RCPO. The problems 'began soon after you decided to stop ba^y-. 
sittinq" for. the recruits and to allow them to take on more of the resportsibi 
ity for running themselves. The RCPO seems to be reluctant to take on the , 
added responsil?ility. He is sending many problems to that he ai?d the otti 
recruits ought to be resolving for themselves^ You haveVbeen sending these 
problems right back to-the RCPO, In fact, you might hav^ been _ sending back 

. more than usual to the RCPO just to give him the idea that he is not,goi"9. to 
get out of work by sending it, .to you'. 




The thing that upsets you mo^t. is that/ the" RCPO appea.rs to be' creating 
poor attitudes in the rest •f the-company with his complaiints about you. 

« . ^ou have noti-ced that something is disturbing,the RCPO]in his recent 
■ interlctiohs wi th . you . Al though. Vou are not su re :?xac«y .wfiat ij i s , you . fee I 
th-at^th*er he 'is ashamed of hisGoor showing recently as in RCPO or he might 
have personal problems.'. He is qi/ieter with you than usual and he ^'ecomes emo- 
tionally upset when you try to .instruct him on how to run the company. When he 
is not upset, he appears to be boredand unconcerned about what you say. some- 
times he ,even claims you never told him some things when you know you have. 

You don't want to return to "babysitting" for the company. You have de- 
cided' to make one more effort to straighten oi>t the RCPO, before you repl ace 
him You don't want to repSlace him because he has good potential and also De- 
cause it might demoralize the company, but you wil 1, if^he doesn t come around 
and show a" better attitude in your next discussion with him. You want to do 
' all you can in this meeting that will increa^se your confidence that the RLHU 
will start taking some positive actions to improve the situation and at the 
same time improve the working refationship between you and hiT- .^^^! '^^^"'^.u, 
efforts in this meetingxlo n(*t leave you confident...that the difficult!^ the 
company will improve, jjou have decided toYelieve the RCPO "fin the spot, and 
you will proceed to do so. . > 

• , V ■ Figure B-2. Role-Play #2 (Contd) ^ 
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' Behavioral Checklist 1 - • 

Concrete ; + gave details, ftf what RCPO's job consists of , ^ 

+ gave details in feedback about how well RCPO is doing his job 

- - + gave details in instruction about what the particular pifoblems 
are and how the RCPO should handle ^them: . 

. (Problems are: (a) RCPO sending CC too many problems; (b) low 
inspection, scores; (c) RCPO .talking against CC; (d) TU has > 
poor attitwdes; (e) DPO needed to establish order^^ff) RCPO 
riot -his usual ;self (quieten, upset, unmotivated) /(^l) communi- 
cation problem- (with RCPO),) 

Rel_evant: + gave gootf 'reasons for allowing the Til to^ run, itself and gave 
good reasons for the benefits of recruit^training in gerieral - 

+ told how running the' Tl) wil 1 benefit- the RCPO ' 

, " . ■ . ' . ■ . . . ■ . 

Human ; \ + admitted that he might not have given enough help with the . . 

cpmpany - - , , - . . ' 

V ^ + showed that he is interested in the recruits' welfare-* 

'* ■•• . ^ • ■ 

+ tpld how he feels about the current problems with the 'company^ 
RTC, etc, 

^ ' ' ^ ■ .., -^^ ' ' ■■ ■■■■ \ 

+ admitted that iie does not know enough about the problems to offer 
the bVst help^ . ' ' ^ ; 

+ '"but the RCPO at ea$e with^some J^umor, expressipf^^ of bersonal 
: . -experiences, etci j . / / 

\ + showed h^M's< interested in RCPO's welfare yf 

+ used words "we", -lus", etc. * * . 

Considerate : + expressed confidence in RCWs ability" to do the job 
■ ' ' ' • *■ 

. + asked RCPO how^he and others feel about him; RTC, problems in 
the company, etc. 



' + did not become unduely angry or yell at RCPO 

+ told RCPO exactly what he planned to da regarding relieving 
him and how he thinks the problems are due to the RCPO's 



performance 

Figure Br2. . Role-Play^2 (C.ontd) ' 
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Set Expectations ; + toldi'RCPO the kinds of problems that he! will fWst Hkely ^ ■ 
— ; ~ encounter at this stage of training ' ' - , , y. 

Time!/:, .+. told RCPO what- are the most importan^ things to do and what nay -be • 
y ; less Important ■ . , . ' ' " ■ ' ' 



' + repeated' important information often, , summarized it well , or other- 
• wise mSHe sure, RCPO would remember it • 

Aware; +'. observed, questioned, and probed the RCPO for his understanding of 
what was said • ' • , 

' Figure B-2. Role-Play #2 (Contd) 
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ROLE-py\YING EXERCISE, 

recruith - ' • * ' . ■ /- 

■ ■ . ■ ^ ■ . .. . f'.-,.-.- 

You are recruit Bradford in TU IBO vhich is in the 2-4 day of training. 
Recently, you have been having some'trouble' rel atirig to your* section leader, 
RPO Sharkey. But this morning he jumped all over you for losing a point in 
the last inspection. When you told him that he showed you the wrong way to 
fold his shirt that he lost the point on, he became extremely offensive' and - 
started shouting and swearing about how he is going to "take care of ydu" If 
you try to blame him for your mistakes. ' Yod felt this was a physical threat 
,that you didn't have to take, so you went to RCPO Cook, to tell him about it. 
Cook right away started -to plan out What to say to^CC Altinan about the inci-v 
dent, which bothered you because you didn'Vwant to^make such a, big thing out 
of it. But since RCPO Cook was so much in Taypr of it,^yl5li fe\\ you had,^ to/go 
.alorig with . his plaV. ' . ' ' • 

So you and RCPO Book got your story together (allow time for this) ai( 
-^ent to see CC Altman.V , . 

You are jusj^ starting to discuss this matter with CC Altman |s th^ role 
play begins. 

' - , «■ ■- . . " . ' - . ' ■ ' 

■■ . V ■ . ' .■ . ■ . ' ■ ' 

CC . ; . ^ ' ,. ' , . ;. 

- ■ ■"•■■■•< " ■ 

You are CC Altman in TU 150. Today, on the 2-4 day of training, you 
were approached by recruit Bradford and RCPO Cook from, your company. Bradford 
reported that his section leader,, Sharkey, maJl-treated him. Up to now, RPO 




motivation in tne irainir\g unit anu pruuauiy wci c. i c^n'""^ ■ ""^ iwi u..v. ^ 
showing on the last inspection. You to.ld the RCPO to watch for these conflicts 
And to try to improve t^e team spirit of the company". You think^hat this^in- 
cident is an example of these conflicts, so you are eager to get to the bo|:tom 
of it. . • ' J ■■ 

You are Jiist. starting to discuss this matter with Bradford and Copk as 
the role play begins. * ^ / 

J 



/9 

RPO'' 



You are RPO Sharkey, section leader in TU 150. YAu woke up^to a particu- 
larly bad day today 2 - 4 day of training. You-were just^out of bed when 
RCPO Cook was all over you about two recrdits in. your sect iVn who lo^t. points 
in the last inspection. He said yqu better get it togeth^ or you won't be 
section l,eader for very long. ,^ 

• Figure B-S. Role- PI ay #3 



' . -'^ . NAVTRAEQUIPCEN IH-308 . ■ . . 

" V " RPO (Contd) • ' . 

<^ RCPO Cook has bfeen out to get yoii since the company fonned and you know 
he is just Ibokjjsg fgr a chalice to get: you. replaced. This i^. espedtrfv^ up- 
Setting because everyone at home is very proud that you were chosien section 
'UaierV In fact, you just receive'd a Utter from your father, who^is veis-y- - 
sick', in which Fte said how-happy i-t would' makeVhim to be ah.le to see you grad- 
uate as an RPO. You have been telling other rehruits that j?6u will get RCPO 
Cook replaced before he does it to you. Most or the other recruits agree with 
'you that ,Cook should be replaced, but say he has the CC under his thumb and 
that you don't have- a chance. You feel that Cook fs^not a good «CPO because of ; 
the abusive. mariner in wh.ich" he deals . with people. ^You rea-lly* think the company 
would -be a lot better off without RCPO Cook.- . ' . . 

RCPO Cook bothered you so nfuch that morning that when r6cruit*.Bradford 
.started to bl^me you for his mess-iip on the inspection., you |)lew.your top and., 
yelled at him a lot. He said he wafsn 't going to take this from you and went 
to see, RCPO Cook about it. ' * • 

You know that Cook and Bradford went to the CC's office to talk. about the 
. matter. Ypu're afraid that the CC won't understand that you're dbinc^ as well 
as anyone else and that RCPO Cook is just using tfils incident to get you. 
You're getting your stbry together as you wait to hear, what happens,* because ; 
you know this could' be-^ a Jreal problem for you. V ' 

( RPO should wait in another room .) Vj* ^ 

'*•■...•*' ■' ■ , . ■ ' 

. ..^ Figure B-3.' Role^Play #3 (Contd) : \ ; 
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. ^ . APPENDIX 

QUESTIONNAI 

The process of evaluation in the inter 
perfonner be ^ng judged and informed about h 
The general purpose, of this study is to det 
tb accomplish some of these evaluation and 
thoughtfiil aftswers tO'the followir^g qU'estic 
will be appreciated . . - * 

Questions for Ea 

1. I felt that, the work load in this;metKc 

a. ' .much too heavy > - 
. b. 'somewhat too heavy <" 
c ; slightly too heavy 
d/ about right / ^ 

■;6. too 1 ight . • 

: 2 Using" this method, I was: 

a. very cbnfLised about *what. I was supp 
.b. somewhat confused about what I was 
" c. ^ slightly confused ^bout what I was 
''d. hot afc-t all>:,coTifused abbut wh'&t I wc 

3, I found this method: 

a.,r very interesting 
i . b» somewhat, interesting 
• c\ "slightly interesting , 

d. boring^ 

* ■ ' ' ' • "' . { 
4 ' Compared^ with oth^r ways for describing 
formance tha.t I can think of,. •►this mett 

' • ' ■ ' ' ' ■» 

4.a. niuch [nore useful 

A . bv more usefuV ' > ^, > 

^c; neither more; nor less useful 

'd. less ^usefuT ' 

e. ?much 1'ess useful ^. - 

5. Please describe. any modifications toi tl 
make it more useful: - 
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, Final. Questions 

The basic idea of using a cortiputer to help a person evaluate videotaped 
interpersonal performance is: 

"^nonsense 

b. • probably not worth pursuing [■ / 

c. possibly not worth pursuing » 
d/ probably worth pur;5uing 

e. definitely Worth pursuing. 

WMch of the 'methods did^you prefer? Why? ^ . - 



What did you like most and' least about , each method?' 
A - most ' ' . 

least ^ „. . 

B - most 

least / 
C T most ' — • ' , 

least 

-Please write below any comments ^which might help us in this study: 
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. ' APPENDIX D ' 

^ . SAMPLE COMPUTER OUTPUTS 

FROM . • /TO _ KEY 

0 MIN 0 SEC q MIN 33 SEC ' 3 , 

;t;**COMMENTS*** ■ ' ' 
NONE „ . 

>|{**SKILLS: EXHIBITED THIS SEGMENT*** • . . 

•FROM . / TO , KEY 

0 MIN- 33 SEC 0 MIN 48 SEC 4 

■***CbMMENTS*** ■ - 

THE CC THREATENED THE RECRUIT ' 

* *)KSK I L L S EXH I BXIEIL JitLST_,.?EGMEN ^^^^^ ' ' 

CONSIDERATE - -. ^ ,^ . ' ' 

■ . . ; FROM " ■ . ' /TO , ■ ~ ■ KEY . 

0 MIN 4-8 SEC 3 WIN 50 SEC '2 

*>K*cdMMENTS***- - - 

NONE - ' : ' ' 

***SKILl's EXHIE<ITED this SEGMENT*** ' 

CONSIDERATE ' ' CLFA^v 

FROM -TO ' KEY . 

3 MIN 50 J5EC .' 5 MIN"00 SEC " 4, 

***COMMENTS*** 

EXPRESSED A LOT OF CONCERN FOR THE RECRUIT AS A PERSON 
AND EXPRESSED CONFIDENCE IN THE RECRUITS ABILITIES 

;K**SKILLS EXHIBITED THI.S SEGMENT*** • 

CONSIDERATE • CLEAR . . \ 

Figure D-1. Method "A" Output 
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CALCULATED 'VTR EFFpcf-H/ENETSS AyERAGr— -3 . 550 
JUDGES SELECTED EFFEC.T-IyE^^ESS AMETjlA'SE— --4 . 000 



SKILL 



OBSERyATIONS 



EFFECTIVENESS 
AVERAGE. • 



CONSrOERATE 
CLEAR 



4.50Q 
4.000 



Figure D-1. Method "A" Output (Contd) 
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EVALUATED SKILL IS CONSIDERATE 



FRQM 
0 Miy 0 SEC 



TO 

0 MIN 55 SEC 



***EiEHAVIORS EXHIBITED THIS SEGMENT*** 

t 

7) WAS' impatient' WITH THE RECRUIT- 



FROM 
0 MIN. 55 SEC 



TO ■ 
1 MIN 38 SEC 



>K**BEHAVIORS EXHIBITED THIS SEGMENT*** 
<?) A BEHAVIOR NOT LISTED" 



TO » 
2 MIN 52 SEC 



FROM 
1 Ml.i^^ 38 SEC 

***BEHAVIORS EXHIBITED THIS SEGMENT*** 



2) EXPRESSED CONFIDENCE IN THE RECRUITS ABILITIES 



FROM 
2 MIN Sj2 SEC 



TO 

5 MIN 00 SEC 



***BEHAVIORS EXHIBITED THIS SEGMENT***/ 

:L,:l) ENCOURAGED THE RECRUIT TO,. ASK QUe/jTIONS 

EVALUATED SKILL. IS "CONSIDERATE" 

OVERALL SKILL .EFFECTIVENESS AVERAGE;/. . . .3.50( 
CALCULATED EFFECTIVENESS AVERAGE ../.♦... 3 . 490 




ke:y 



KEY 
4 



KEY 
3 



KEY 
4 



Figure D 



-2. MetSo/d "B" 



Output 
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EVALUATE^D SKILL IS CLEAR ' ' 

• '""from' ^ TO KEY 

0 MIN 0 SEC ■ 0 MIN 37 SEC 3 

#«*BEHAVIORS EXHIBITED THIS SEGMENT*** 
,14) WAS VAGUE 

"'I ^ _ « _ — — — — — — 

FROM TO ■ ' ' KEY 

0 MIN 37 SEC . 2 MIN 15 SEC 4 

***BEHAVIORS EXHIBITED THIS SEGMENT*** 
10) GAVE SPECIFIC DETAILS 



FROM TO KEY 

2 MIN 15 SEC 3 MIN 44 SEC 



***BEHAVIORS EXHIBITED THIS SEGMENT**)!^ 



13) ' GAVE REASONS FOR THE THINGS RECRUITS DO A"pRTir 
18) A BEHAVIOR NOT LISTED • 

~~""""""fR07""""""^""^"'t0 ' , KEY 

3 MIN 44 SEC ■ 5 MIN 00 SEC 4 

>K**BEHAVI0RS EXHIBITED THIS SEGMENT*** 

14) ,WAS VAGUE 

EVALUATED SKILL IS "t^LEAR" ■' 



OVERALL SKILL EFFECTIVENESS AVERAGE .... 4 . 000 
CALCULATED »FFECTI VENESS AVERAGE .3.400 

JUDGE'S OVERALL EFFECTIVENESS AVERAGE ..... 4 . 000 



Figure D-2. Method "B" Outptut (Contd)^ 
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FROM - TO _ 

0 MIN a SEC 0 MIN 33 SEC 

>K##BEHAVIORS' EXHI^ITEH THIS SEGMENT*** 
14) WAS VAGUE 

9) A, CONSIIiERATE BEHAVIOR NOT LISTED, 

FROM TO. J - ■'[.■. 

, Q MIN 33 SEC 1 Ml^i SEC 

***BEHAVIORS EXHIBITED THIS SEGM£Nt*** • / . 

:L0) gave SPECIFIC DETAILS ^ 

18) A CLEAR BEHAVIOR NOT LISTED ^ 

FROM TO ' . • ' 

1 MIN 47 SEC 2 MI^^ 59 SEC ■ • 

■ • ' » 

***BEHAVIORS EXHIBITED THIS SEGMENT*** 
■ « ' ^ 

•5) SHOW^i DISINTEREST IN THE RECRUIT'S FEELJNGS 

6) BELITTLED THE RECRUIT'S ABILITIES ^^'^ 

FROM ■ . TO . • . 

-2 MIN 59 SEC ' 5 MIN 00 SEC . 

**^BEHAVI0RS EXHIBITED THIS SEGMENT*** 



18) A CLEAR BEHAVIOR NOT LISTED 
JUDGE'S OVERALL EFFECTIVENESS AVERAGE..' 
SKILL .• AVERAGE 

- V 

CONSIDERATE 2.000^ 
CLEAR . 3.000': 

Figure D-3. Method "C" Output 
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EVALU(!^tED SKILL IS CLEAR \ . 



r. 





.FROM TO - KEY 

0 MIN 0 SEC , .. 1 MIN 51, SEC . 

*««BEHAVIORS EXHIBITED THIS SEGMENT*** 
10) GAVE SPECIFIC DEJAILS 

FROM ■ • * TO ^EY » 

. 1 MIN 51 SEC • •• 2 MIN 34 SEC . 2 . 



***IieftAVIORS EXHIBITED THIS SEGMENT:(c** • 

r ;. } ^ ■ ,.. ' ■ . a,' A- 

:L4> was VAGUE - . , ^ 

16) FAILED TO. QUESTION THE RECRUITS FOR THEIR UNDERSTANDING,:. 

#**BEHAVIOR NOT LISTED*** 

INCORRECTLY INFERS ATTITUDE FROM BEHAVIIjiR 

FROM " TOf KEY 

2 MTN 34 SEC 3 MIN 49 SEC 3 . , 

***BEHAVIORS EXHIBITED vTHIS SEGMENT*** 
10) GAVE SP^IFIC DETAILS . 

13) GAVE REASONS TO SHOW -RELEVANCE OF WHAT WAS SAID 



FROM - TO , I<e:y 

3 MIN 49 SEC 5 MIW 00 SEC ., 3 

***BEHAVIORS EXHIBITED THIS SEGMENT*** • 

■ ■ * - 

10) GAVE SPECIFIC DETAILS 

13) GAVE REASONS TO SHDW RELEVANCEOF WHAT W^S SAID 
■***BEHAVIOR NOT LIST^i*** 
NOT CLEAR IN QUESTIONING 



SKILL AVERAGE « 



CLEAR ■ 3 

CALCULATED 5 MINUTE VIDEOTAPE AVERAGE . ... 2 . 940 



Figure D-4. New.Method Outputw, ^r. 
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EVALUATED SKILL. IS CONSIDERATE 



FROM 
0 MIN 0 SEC 



TO 

1 MIN 02 SEC 



ke:y 



**^BEHAVIORsTxHlBITED THIS SEGMENT*** 

/) WAS^ IMPATIENT WITH THE RECRUIT 
'f?<x^ff^-^E RECRUIT'S' NAME 

FROM • TO 

i MIN 02 SEC 1 MIN 47 SEC 

*^*'ErEHAVIORS EXHIBITED . THIS SEGIMENT***' 

6)^J!BELITTLED THE RECRUIT'S ABILITIES 



KEY 



FROM , 
' 1 MIN 47, SE 



TO 

3 MIN 33 SEC 



KEY 
3 



>K**BEHAUI0RS EXHIBITED THIS SEGMENT*** 
:10), ASKED T'HE RECRUIT HOW HE/SHE FELT ABOUT THE SITUATION 
*H<*BEHAVIOR ,NOI LISTED*** 
SEEMED DETACHED 



FROM 
3 MIN 33 SEC 



' . TO , 
5 MIN 00 SEC 





>K**FI:HAVI0R^ EJeHI&ITED THIS SEGMENT*** 
7) WAS IMPATIENT WITH THE RECRUIT 
***BEHAVIOR NOT LISTED*** 
POOR LIS'TENING •; 



SKILL , AVEFiAGE 

CONSlDeRA'TE 2.000 ' \.. 

CALCULATED 5 MINUTE VIDEOTAPE AVERAGE 2 . 120, 

JUDGE '^' OVERALL EFFECTIVENESS AVERAGE. . . . ..2.700 

Figure D-4. New Method Output (Contd) 
' '■ ' . 45/46 
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